Skip to main content

Python Development

Expert Product Development with Python Online

Introduction​

Introducing Product Development with Python

  • From organic grown scripts to product
  • Training Goal
  • Course Roadmap
  • Modules Overview

An Essential Tour of Python​

An advanced tour of Python's essential features

  • Name Binding
  • Compiled or Interpreted?
  • Global vs Local
  • LEGB Scoping Rules
  • Argument Passing
  • Mutability
  • Function Arguments
  • Special Parameters
  • Object in Python
  • First Class Objects
  • Decorator
  • Lambda, Iterable and Comprehension
  • F-strings Virtual Environment

Debugging​

Efficient debugging tools and techniques in Python

  • Builtins Introspection
  • inspect
  • inspect stack frame
  • debugging playground
  • starting pdb
  • pdb commands
  • interactive session
  • Breakpoints
  • Inspect and modify
  • pdb Cheat Sheet
  • Switching debugger
  • Visual debugger
  • The Scientific Method
  • Going Further

Pythonic OOP​

Advanced Pythonic object-oriented programming techniques

  • Class 101
  • Class methods
  • Alternative Constructors
  • Static Methods
  • Virtual methods
  • Private Name Mangling
  • Instance variables vs getter/setter
  • property
  • property setter and deleter
  • Interface and ABC
  • ABC in Python
  • ABC and collections
  • Multiple Inheritance
  • MRO
  • Multiple Inheritance and Interfaces
  • Mixins
  • slots
  • Bonus Annex

Modules and Package​

A deeper understanding of modules and packages and how to use them

  • import
  • import under the hood
  • module caching
  • from…import
  • import *
  • name
  • packages
  • import from Package
  • Running modules from the command line
  • logical namespace
  • practical considerations.

Code as Docs​

Ways to document and structure code for peers and code users

  • Documentation strings
  • documenting API
  • type hints
  • mypy
  • code formatting
  • black
  • isort

Distribution Package​

How to create a distribution package

  • introduction
  • Really Simple Script (rss)
  • toward productization
  • Python Project Layout
  • pyproject.toml
  • Configuring Metadata
  • Distribution Package Metadata
  • classifiers
  • package dependencies
  • Controlling Package Contents
  • Prod/Dev Environment
  • Configuring Black and Isort
  • Python Application
  • Testing your Distribution Package
  • Install for Source
  • Building
  • Going Further
  • Bonus Annex

Testing​

Introducing pytest and common tools and techniques used for testing python products

  • Introduction
  • Testing Considerations
  • Testing Strategy
  • The Four-Phase Test pattern
  • Fake and Test Doubles
  • Doctest for e2e Testing
  • Doctest and docstring
  • Pytest Architecture
  • A Simple Test
  • A Failing Test
  • Test Discovery
  • Grouping Tests in a Class
  • Testing that an Exception is Raised
  • Skipped Tests and Expected Failures
  • Parametrization
  • Test Summary Report
  • Dropping into pdb
  • Test Fixtures
  • User-Defined Test Fixtures
  • Mocking
  • Monkeypatch Fixtures
  • Monkeypatch and Mocking Fixtures
  • pytest Good Practices

Logging​

The Python logging module

  • Logging: An overview
  • Loggers
  • Log level
  • Handlers
  • Formatters
  • Loggers hierarchy
  • Filters
  • Logging Workflow
  • Practical Consideration
  • Config by Coding
  • Logging on an Embedded System
  • Additional Resources
  • Bonus Annex

Command Line Interface​

How to create a sound command line interface in Python

  • Introduction
  • CLI Design Guidelines
  • Exiting Python Programs
  • Handling Signal
  • Custom Signal Handler
  • Dealing with Ctrl-C
  • Parsing Argument
  • Positional Arguments
  • Usage and Help
  • Option Arguments
  • Options With and Without Values
  • Optional Values
  • nargs
  • Description, Epilog, Help
  • Prefix, Choices, Required
  • Click
  • Command Examples
  • From argparse to click
  • Click Path and File
  • Command Group
  • Invoking Sub-commands
  • Improving User Experience
  • Tqdm Progress Bar
  • Rich

Quality Assurance​

Common tools and techniques to assess and improve the quality of your python product

  • Static Analysis and Type Checkers
  • flake8
  • pylint
  • Security Auditing
  • Practical Considerations
  • Integrating Static Analysis Tools
  • Type checkers
  • Code Coverage
  • Coverage Report
  • Coverage Area
  • Branch Coverage
  • Mutation Testing
  • mutmut Output
  • Analysing mutmut Report
  • Tips and Tricks

Concurrency​

An overview of concurrency and available modules from the standard library

  • What is concurrency
  • CPU vs I/O bound tasks
  • Concurrent Programming Models
  • Concurrency in Python
  • Concurrency Modules
  • Modules Summary

Threading​

Introducing the threading module to implement concurrency using OS-threads

  • Introduction
  • Thread Creation
  • Thread Creation Using Class
  • Thread and Program Termination
  • Joining Thread
  • Thread Local Data
  • Race Condition
  • Synchronizations Objects
  • Lock Object
  • Event Object
  • Condition Object (waiters)
  • Condition Object (notify)
  • Communicating between Threads
  • Working with the GIL
  • Bonus annex

Multiprocessing​

Introducing the multiprocessing module to implement concurrency using OS processes

  • Introduction
  • Starting a New Process
  • Start Methods
  • Choosing a Start Method
  • Inheriting Resources
  • Join, Terminate and Exit Code
  • Inter-Process Communication
  • Pipe
  • Queues
  • Shared Memory
  • Manager
  • Process Pool
  • Task Offload Methods
  • Bonus Annex

Executors and Futures​

Some of the high-level constructs provided by the concurrent.futures module

  • Introduction
  • Executor Object
  • Thread Executor Pool
  • Process Executor Pool
  • Future
  • Future and Deadlocks
  • Customizing Future Execution

AsyncIO​

Introducing the asyncio module to implement concurrency using Python coroutines

  • Introduction
  • Async IO architecture
  • async, await and coroutines
  • asyncio.run() Error
  • asyncio Tasks
  • Awaitable Objects
  • Cancellation and Shielding
  • Await on Multiple Objects
  • await with Timeout
  • Monitor Completion
  • await for the Earliest Result
  • Synchronization Objects
  • Queues
  • Subprocess
  • Interacting with Subprocess
  • Many Other Features

Interfacing Native Resources​

How to interface native resources in Python

  • Introduction
  • Interfacing Native resources
  • Using the Standard library
  • ctypes
  • numba
  • Embed C with CFFI
  • Building and Running CFFI
  • Compiling from C Header and Source Files
  • Building from a Shared Object File
  • Pointers and Structures
  • CFFI Build Script
  • ffi.new
  • cdef() – practical considerations
    • To distribute CFFI with PyInstaller
    • Cython
    • Cython Language
    • Primes in Python
    • Primes in Cython
    • Compile with Cython
    • Comparing Numba, Cython, and cffi
    • Compare the Speed
    • Summary

Python Wheels​

Deep dive into how to build Python wheels to integrate binary extension modules

  • Introduction
  • Package with Native Resources
  • Python Wheels
  • Pure vs Binary Wheels
  • Building binary wheels
  • Packaging Cython Extension
  • Packaging CFFI Extension
  • Cross-Distribution Linux Wheel
  • Manylinux
  • Manylinux Docker Images
  • Build a manylinux Wheel Manually
  • Supporting Multiple Platforms
  • Cross-Building Wheels
  • cibuildwheels

Profiling​

Common python tools and techniques to spot performance bottlenecks

  • Introduction
  • Profilers Technology
  • Top-Down Strategy
  • Timing Execution
  • Benchmark
  • cProfile
  • Sort by cumtime
  • Visualizing Report
  • Line profiling
  • Gathering Line Profiling Data
  • Line Profiling Result
  • Profiling Memory Usage
  • Line-by-Line Memory Profiling
  • Memory Profiling Tips & Tricks Scalene
  • Scalene CLI report
  • Scalene CPU usage
  • Scalene Memory Usage
  • Optimization Loop
  • Micro-benchmarking
  • Choosing your Profilers
  • Bonus Annex

Shipping Your Product​

How to ship a Python Project

  • Overview
  • Distribution Package Lifecycle
  • PyInstaller
  • Dist Folder
  • PyInstaller Option
  • Source Distribution
  • Publishing Locally
  • Publishing on PyPI
  • testPyPI
  • Upload Preparation
  • Upload to testPyPI
  • Checking Project Page
  • Testing our Package
  • Uploading on PyPI
  • Automating Build
  • Going further

Product Documentation (Optional Topic)​

How to document your product and generate the documentation with Sphinx

  • Documenting API
  • Sphinx Overview
  • Sphinx Quickstart
  • Sphinx Configuration
  • Documentation Layout
  • reST markup
  • Generating Documentation
  • Writing Good Documentation

Security Considerations (Optional Topic)​

Some security considerations related to our Python product

  • Shopping on PyPI – The Challenges
  • Typosquatting Attacks
  • Looking for Candidates
  • Project Check List
  • Package Evaluation
  • Package Vulnerabilities Report
  • Securing Python Application
  • Python Security Cheat Sheet
  • Python Security Best Practice
  • Code Obfuscation
Last updated on by Viraj